Full-wave analysis of dielectric waveguides at a given frequency

New variational formulation to compute propagation constants is proposed. Based on it, vector finite element method is proved to exclude spurious modes provided finite elements possess discrete compactness property. Convergence analysis is conducted in the framework of collectively compact operators. Reported theoretical results apply to a wide class of vector finite elements including two families of Nedelec and their generalization, the hpedge elements. Numerical experiments fully support theoretical estimates for convergence rates.